27 research outputs found
Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmark
Psychological research results have confirmed that people can have different
emotional reactions to different visual stimuli. Several papers have been
published on the problem of visual emotion analysis. In particular, attempts
have been made to analyze and predict people's emotional reaction towards
images. To this end, different kinds of hand-tuned features are proposed. The
results reported on several carefully selected and labeled small image data
sets have confirmed the promise of such features. While the recent successes of
many computer vision related tasks are due to the adoption of Convolutional
Neural Networks (CNNs), visual emotion analysis has not achieved the same level
of success. This may be primarily due to the unavailability of confidently
labeled and relatively large image data sets for visual emotion analysis. In
this work, we introduce a new data set, which started from 3+ million weakly
labeled images of different emotions and ended up 30 times as large as the
current largest publicly available visual emotion data set. We hope that this
data set encourages further research on visual emotion analysis. We also
perform extensive benchmarking analyses on this large data set using the state
of the art methods including CNNs.Comment: 7 pages, 7 figures, AAAI 201
Robust Image Sentiment Analysis Using Progressively Trained and Domain Transferred Deep Networks
Sentiment analysis of online user generated content is important for many
social media analytics tasks. Researchers have largely relied on textual
sentiment analysis to develop systems to predict political elections, measure
economic indicators, and so on. Recently, social media users are increasingly
using images and videos to express their opinions and share their experiences.
Sentiment analysis of such large scale visual content can help better extract
user sentiments toward events or topics, such as those in image tweets, so that
prediction of sentiment from visual content is complementary to textual
sentiment analysis. Motivated by the needs in leveraging large scale yet noisy
training data to solve the extremely challenging problem of image sentiment
analysis, we employ Convolutional Neural Networks (CNN). We first design a
suitable CNN architecture for image sentiment analysis. We obtain half a
million training samples by using a baseline sentiment algorithm to label
Flickr images. To make use of such noisy machine labeled data, we employ a
progressive strategy to fine-tune the deep network. Furthermore, we improve the
performance on Twitter images by inducing domain transfer with a small number
of manually labeled Twitter images. We have conducted extensive experiments on
manually labeled Twitter images. The results show that the proposed CNN can
achieve better performance in image sentiment analysis than competing
algorithms.Comment: 9 pages, 5 figures, AAAI 201
Cultural Diffusion and Trends in Facebook Photographs
Online social media is a social vehicle in which people share various moments
of their lives with their friends, such as playing sports, cooking dinner or
just taking a selfie for fun, via visual means, that is, photographs. Our study
takes a closer look at the popular visual concepts illustrating various
cultural lifestyles from aggregated, de-identified photographs. We perform
analysis both at macroscopic and microscopic levels, to gain novel insights
about global and local visual trends as well as the dynamics of interpersonal
cultural exchange and diffusion among Facebook friends. We processed images by
automatically classifying the visual content by a convolutional neural network
(CNN). Through various statistical tests, we find that socially tied
individuals more likely post images showing similar cultural lifestyles. To
further identify the main cause of the observed social correlation, we use the
Shuffle test and the Preference-based Matched Estimation (PME) test to
distinguish the effects of influence and homophily. The results indicate that
the visual content of each user's photographs are temporally, although not
necessarily causally, correlated with the photographs of their friends, which
may suggest the effect of influence. Our paper demonstrates that Facebook
photographs exhibit diverse cultural lifestyles and preferences and that the
social interaction mediated through the visual channel in social media can be
an effective mechanism for cultural diffusion.Comment: 10 pages, To appear in ICWSM 2017 (Full Paper